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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) [g) Claim(s) 1-13 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |El Claim(s) 7-6 and 8 is/are rejected. 

7) K) Claim(s) Sand 7-13 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) £3 The specification is objected to by the Examiner. 

10) IKI The drawing(s) filed on 31 August 2000 is/are: a)S accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)Q All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3,Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 



1) K| Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1449) Paper No(s) 4 . 



4) O Interview Summary (PTO-413) Paper No(s). 

5) □ Notice of Informal Patent Application (PTO-1 52) 

6) Q Other: 



U.S. Patent and Trademark Office 
PTO-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 3 



Application/Control Number: 09/652,820 Page 2 

Art Unit: 2623 

DETAILED ACTION 
Specification 

1 . The abstract of the disclosure is objected to because of undue length and "The steps" 
should be changed to "the step" on line 14 and "afirst" should be changed to "a first" on line 24. 
Correction is required. See MPEP § 608.01(b). 

2. The disclosure is objected to because of the following informalities: "09/653,023" should 
be inserted after "Serial No." on page 1, line 5; "9/1/00" should be inserted after "filed on" on 
page 1, line 7; "Rj" should be changed to "R'" on page 8, line 14; and "(smoothed" should be 
changed to "(smoothed)" on page 9, line 3. The first paragraph on page 3 is almost identical to 
the preceding paragraph on page 2. It is suggested to delete the first para, on page 3. 

Appropriate correction is required. 



Double Patenting 

3. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. See In re Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. 
Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 
F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 
1970);and, In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) may be used to 
overcome an actual or provisional rejection based on a nonstatutory double patenting ground 
provided the conflicting application or patent is shown to be commonly owned with this 
application. See 37 CFR 1.130(b). 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 
CFR 3.73(b). 
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4. Claim 1 is provisionally rejected under the judicially created doctrine of obviousness- 
type double patenting as being unpatentable over claim 1 of copending Application No. 
09/653,023 in view of Aguiar et al. (Factorization as a Rank 1 Problem) and in further view of 
Irani (Multi-Frame Optical Flow Estimation Using Subspace Constraints). 

Regarding claim 1 of the instant application, claim 1 of 09/653,023 covers equivalent 
subject matter except for the limitations regarding rank-1 factorization and dividing the 
successive images into smoothing windows. Using rank-1 factorization rather than rank-3 factor 
matrices is known in the art as evidenced by Aguiar et al. (Abstract, lines 1-12). Therefore, it 
would have been obvious to one of ordinary skill in the art at the time of the invention to have 
modified the rank-3 factor matrices of claim 1 of 09/653,023 in view of Aguiar et al. in order to 
allow the use of fast algorithms (Abstract, lines 11-13). Dividing the images into smoothing 
windows is also known in the art as evidenced by Irani (Sect. 3.2, Para. 1). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time of the invention to have modified 
claim 1 of 09/653,023 in view of Irani to include dividing into smoothing windows in order to 
simplify computations (Sect. 3.2, Para. 2). 

This is a provisional obviousness-type double patenting rejection. 

Claim Objections 

5. Claim 5 is objected to because of the following informalities: "}" should be added before 
"where" on page 18, line 21 and "matrix with" should be changed to "matrix" on page 19, line 9. 
Appropriate correction is required. 
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6. Claim 8 is objected to because of the following informalities: "cameras optical" should 
be changed to "camera's optical" on page 21, line 2. Appropriate correction is required 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. Claims 1-6 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Oliensis 
et al. {Structure from Motion using Points, Lines, and Intensities) in view of Aguiar et al. 
{Factorization as a Rank 1 Problem). 

Regarding claim 1, Oliensis et al. ("Oliensis") discloses a method for recovering 3D 
scene structure and camera motion from image data obtained from a multi-image sequence 
(Sect.l, para. 1). Oliensis discloses determining image data shifts for each successive image 
with respect to the reference image, the shifts being derived from the camera translation and/or 
rotation from the reference perspective to the successive different perspectives (Sect. 3.1). 
Oliensis further discloses constructing a shift data matrix that incorporates the image data shifts 
for each image and calculating a factorization of the shift data matrix using SVD corresponding 
to the 3D structure and size of the camera motion (Sect. 1, para. 7), recovering the direction of 
camera motion from the first vector corresponding to the 3D structure by solving a linear 
equation and recovering the 3D structure by solving a linear equation using the recovered camera 
motion (Sect. 3.5). Oliensis discloses that it is known to divide the successive images into 
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smoothing windows (Sect. 3.6, para. 5). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to applied to smoothing windows to the 
successive images in order to apply the smoothing constraint. Oliensis does not recognize 
calculating a rank-1 factorization from the shift data matrix. However, Aguiar et al. ("Aguiar") 
teaches that it is known to use rank-1 factorization rather than rank-3 (Abstract, lines 1-12). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to have modified the rank-3 disclosed by Oliensis to use rank-1 factorization, as taught 
by Aguiar, in order to allow the use of fast algorithms (Abstract, lines 11-13). 

Regarding claim 2, Oliensis discloses computing a first projection matrix (Sect. 3.3, para. 
7), recovering camera rotation vectors from the shift data matrix and the first projection matrix 
(Sect. 3.5). Oliensis further discloses computing a second projection matrix (Sect. 3.3, para. 7) 
and recovering the direction of camera translation using the shift data matrix, the reference 
image, the second projection matrix and the recovered camera rotation vectors (Sect. 3.5). 

Regarding claim 3, Oliensis discloses recovering the 3D structure from the shift data 
matrix, the reference image, the recovered camera rotation vectors, and the recovered direction 
of translation vectors (Sect. 3.5). 

Regarding claim 4, Oliensis discloses recovering the rotations of the camera between 
each successive image and warping all images in the sequence toward the reference image while 
neglecting the translations (Sect. 3.1, para. 2). 

Regarding claim 5, Oliensis discloses computing H and D C h or "A C h" (Sect. 3.5. para. 4) 
where H is a (N P -3) x N P matrix defined so that HH T is the identity matrix and H annihilates the 
three vectors vj/ x ,\|/y, 1|/ 2 (Sect. 3.5, para. 1) where the three vectors are computed from the 
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reference image as recited on page 18, line 21 of Claim 5 (Sect. 3.4, bottom of page 601) where 
r (l) (x,y), r (2) (x,y), and r (3) (x,y) are defined as recited on page 19, line 1 of Claim 5 (Sect. 3.2). 
Oliensis discloses A as a shift data matrix that gives the difference in intensities between each 
successive image and the reference image (Sect. 1, para. 7) and is a (N t -1) x N p matrix with 
entires AC (Sect. 3.5) where AI n ' is the change in (smoothed) intensity with respect to the 
reference image and with respect to the reference image and with no smoothing AI n l = I n ' - In° 
(Sect. 3.4, Para. 2) where Nj is the number of images (Sect. 2, para. 2), N p is the number of 
pixels and where V denotes the i-th image with i=0. 1 . . . ,Ni - 1 (Sect. 2, para. 5) where In 1 = V(p n ) 
denotes the image intensity a the n-th pixel position in I 1 where 1° is the reference image where x 
and y are the image coordinates of the pixel position and p=(*>y) (Sect. 2, para. 5) and where Ach 
= C 1/2 AH T where C is a constant matrix (Sect. 3.5) and where the notation {V} used to denote a 
vector with elements given by V a (Sect. 2, para. 6). 

Regarding claim 6, Aguiar discloses computing a rank-1 factorization -Ach = M (1) S (1)T 
where M (1) and S (1)T are vectors corresponding to motion and structure (Sect. 3 5 page 180). 

Regarding claim 8, Oliensis discloses setting Z n _1 as constant within each window, where 
Z is the depth from the camera to a 3D scene along the camera's optical axis (Sect. 3.6, para. 5) 
and listing each of the pixels so that those in the k-th smoothing window have sequential indices 
(Sect. 2, para. 5). 
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Allowable Subject Matter 



9. Claims 7 and 9-13 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 



1 0. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Wang et al., "A SVD Decomposition of Essential Matrix with Eight Solutions for the 
Relative Positions of Two Perspective Cameras" 

Diamantaras et al. 5 "Camera Motion Parameter Recovery Under Perspective Projection" 

and 

Kung et al., "An SVD Approach to Multi-Camera-Multi-Target 3D Motion-Shape 
Analysis." 



1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Virginia M Kibler whose telephone number is (703) 306-4072. 
The examiner can normally be reached on Mon. - Thurs. 8:00 - 5:30 and every other Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone numbers for the 
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organization where this application or proceeding is assigned are (703) 872-9314 for regular 
communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 306-0377. 

VK 

May 16, 2003 
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SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 




